This is a basic Longformer model specifically designed for Russian, supporting a context length of up to 4096 tokens. It is initialized based on the weights of blinoff/roberta-base-russian-v0 and fine-tuned on a Russian book dataset.
Large Language Model
Transformers